Colour Indexing Across Illumination

Because the colours in an image convey a lot of information, almost all image database systems support colour content queries. Unfortunately colour based queries do not always return the images that were sought even although there is a good colour match. Such failures are easily explained. We as human observers do not see raw image colours but rather make an interpretation of the colours in an image. Our interpretation allows us to decouple the intrinsic colour of the objects and surfaces, captured in an image, from the colour of the illumination. An indoor picture with a yellowish colour cast is interpreted as just that, we do not think that all the objects in the scene are more yellow than they usually are. In contrast, image database systems generally make no such comparable interpretation. In this paper we set forth an experimental study that attempts to quantify the nature and magnitude of the illumination colour problem. We are interested in measuring how image colours depend on illumination and how this dependency might be removed. Our work based on a small, but accurately calibrated, image database comprising 11 colourful objects imaged under 4 typical household lights. Because illumination colour impacts so dramatically on image colours, querying this dataset by colour-content delivers very poor indexing. To improve indexing performance, the illumination bias in images needs to be removed. This is done by applying an appropriate mapping to the image colours (e.g. a reddish cast can be removed by reducing the redness at each pixel). We found that mapping image colours based on a measure of the illuminant results in good indexing. However, when the mapping depends on both scene content and illumination together, the indexing performance is even better still. This is a surprising result since it is accepted doctrine that a change in illuminants should result in a systematic change in image colours and this change should effect all images equally (scene content should not add any useful information). Of course if illumination colour depends on scene content then it will be difficult to measure since the spectral statistics of the scene are also unknown. If measurement is difficult, estimation (using a colour constancy algorithm) must be more difficult still.